Papers with language and vision models
Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing language and vision models can be used for language understanding in 3D environments . however, existing models lack specific properties and biases that limit their performance . |
| Approach: | They propose a framework that uses a camera to generate images from different viewpoints and evaluate them in terms of their similarity to natural language descriptions. |
| Outcome: | The proposed model performs poorly on most canonical views and fine-tunes using hard negative sampling and random contrasting yields good results even under conditions with little available training data. |
Rogue Scores (2023.acl-long)
Copied to clipboard
| Challenge: | Critical evaluation decisions and parameters are routinely omitted, making most reports irreproducible . Thousands of papers use nonstandard evaluation packages with software defects that produce incorrect scores. |
| Approach: | a systematic review of over two thousand papers using a popular metric called ROUGE finds errors . critical evaluation decisions and parameters are routinely omitted, making most reported scores irreproducible . a large number of ROUGEE model evaluation scores have been incorrectly computed . |
| Outcome: | a systematic review of over two thousand papers finds that ROUGE scores are incorrect . the metric is widely used in machine learning and is inconsistent with human evaluations . |
ContraSim – Analyzing Neural Representations Based on Contrastive Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing similarity measures perform mediocrely on standard benchmarks . |
| Approach: | They develop a similarity measure based on contrastive learning that learns a parameterized measure by using both similar and dissimilar examples. |
| Outcome: | The proposed measure achieves much higher accuracy than previous similarity measures . it is more suitable for the analysis of neural networks, revealing new insights . |